E. GORDIENKO and O. HERNÁNDEZ-LERMA (México) AVERAGE COST MARKOV CONTROL PROCESSES WITH WEIGHTED NORMS: VALUE ITERATION
نویسنده
چکیده
This paper shows the convergence of the value iteration (or successive approximations) algorithm for average cost (AC) Markov control processes on Borel spaces, with possibly unbounded cost, under appropriate hypotheses on weighted norms for the cost function and the transition law. It is also shown that the aforementioned convergence implies strong forms of AC-optimality and the existence of forecast horizons.
منابع مشابه
E. GORDIENKO and O. HERNÁNDEZ-LERMA (México) AVERAGE COST MARKOV CONTROL PROCESSES WITH WEIGHTED NORMS: EXISTENCE OF CANONICAL POLICIES
This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the exist...
متن کاملDiscrete-time Markov control processes with discounted unbounded costs: Optimality criteria
We consider discrete-time Markov control processes with Borel state and control spaces, unbounded costs per stage and not necessarily compact control constraint sets. The basic control problem we are concerned with is to minimize the infinite-horizon, expected total discounted cost. Under easily verifiable assumptions, we provide characterizations of the optimal cost function and optimal polici...
متن کاملThe Policy Iteration Algorithm for Average Reward Markov Decision Processes with General State Space
The average cost optimal control problem is addressed for Markov decision processes with unbounded cost. It is found that the policy iteration algorithm generates a sequence of policies which are c-regular (a strong stability condition), where c is the cost function under consideration. This result only requires the existence of an initial c-regular policy and an irreducibility condition on the...
متن کاملAcceleration Operators in the Value Iteration Algorithms for Average Reward Markov Decision Processes
One of the most widely used methods for solving average cost MDP problems is the value iteration method. This method, however, is often computationally impractical and restricted in size of solvable MDP problems. We propose acceleration operators that improve the performance of the value iteration for average reward MDP models. These operators are based on two important properties of Markovian ...
متن کاملSimplified policy iteration for skip-free Markov decision processes
We describe and analyse a new simplified policy iteration type algorithm for finite average cost Markov decision processes that are skip-free in the negative direction. We show that the algorithm is guaranteed to converge after a finite number of iterations, but the computational effort required for each iteration step is comparable with that for value iteration. We show that the analysis can b...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2007